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Claims 

1 . Method for multimodal interaction between an user and a terminal 
comprising an input unit, an output unit and a browser application, 
which is capable of interpreting a mark-up language, wherein the 
interaction takes place over the input unit, and/or the output unit and 
by using the browser application and the mark-up language, 

and wherein, the mqrk-up language comprises an extension of 

multimodal meta tags for multimodal interactions, 

and wherein the method comprises the steps of: 

interpreting multimodal meta tags from the mark-up language for 

controlling the multimodal interactions and 

processing data of the multimodal interaction with respect to the 

multimodal meta tags by using at least one input and/or output 

processing application. - . . , ..... . . 

2. Method according to claim 1, wherein the browser application is 
controlled corresponding to said multimodal meta tags and/or a 
communication between the browser application and the input and/or 
output processing applications is controlled corresponding to said 
multimodal meta tags. 

3. Method according to claim 1 , wherein said at least one input and/or 
output processing application is one of the following applications: 

- handwriting recognition application 

- speech recognition application 
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- eye movement recognition application 

- speech generation application 

- pointing recognition application. 

4. Method according to claim 1 , wherein said input and/or output 
processing application is provided at the terminal. 

5. Method according to claim 1, wherein said input and/or output 
processing application is realised as an application having a distributed 
architecture. 

6. Method according to claim 1, comprising by rules determining the 
handling of a plurality of multimodal interactions being related to each 
other. 

7. Terminal for multimodal interaction between an user and said terminal 
comprising an input unit, an output unit and a browser application, 
which is capable of interpreting a mark-up language, wherein the 
interaction takes place over the input unit, and/or the output unit and 
by using the browser application and the mark-up language, 

and wherein said used mark-up language comprises an extension of 
multimodal meta tags for multimodal interactions, 
said multimodal meta tags from the mark-up language are interpreted 
at the terminal for controlling the multimodal interactions, and said 
terminal comprises at least one input and/or output processing 
application for processing data of thp multimodal interaction with 
respect to the multimodal meta tags. 

8. Terminal according to claim 7, wherein means are provided for 
determining and evaluating time information of said multimodal 
interaction. 

9. Browser application used for multimodal interaction between an user 
and a terminal comprising an input unit, an output unit, at least one 
input and/or output processing application for processing data of the 
multimodal interaction, and said browser application, 

wherein the multimodal interaction takes place over the input unit. 
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and/or the output unit and by using said browser application and a 
mark-up language comprising an extension of multimodal meta tags 
for multimodal interactions, 

and wherein said browser application interprets the mark-up language 
with said multimodal meta tags, 

said browser application is controlled corresponding to said multimodal 
meta tags and/or a communication between the browser application 
and the input and/or output processing applications is controlled 
corresponding to said multimodal meta tags. 

10. Mark-up language, in particular used for representing of information of 
a mark-up language document in connection with q browser 
application, wherein the mark-up language comprises meta tags 
specifying properties and values of said properties of the mark-up 
language document, and wherein said mark-up languages further 
comprises multimodal meta tags for controlling the multimodal 
interactions and processing data of the multimodal interaction with 
respect to the multimodal meta tags by using at least one input and/or 
output processing application. 



